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Our  objective  in  this  research  is  to  allow  teams  that  are  physically  separated  to  do  detailed  design 
work  on  large-scale,  3D  projects.  The  specific  task  is  to  enable  multiple  users  to  stand  at  each  of 
two  (or  more)  Virtual  Workbenches  at  remote  locations  and  interact  effectively  for  design 
generation.  Specific  tasks  completed  include: 

•  Organized  and  segmented  the  new  Navy  Arsenal  Ship  model 

•  Developed  off-axis  stereoscopic  display  and  basic  tracking  mechanisms 

•  Developed  an  object  data  organization 

•  Generated  nearly  1  ,(XX)  3D  objects  and  classified  them  using  the  data  organization  format. 

•  Worked  on  several  interaction  paradigms,  many  of  which  are  implemented  in  the  design 
environment. 

•  Developed  a  networking  structure 

•  Put  together  and  are  demonstrating  a  prototype  design  environment. 

In  the  coming  year  we  will  further  study  issues  of  collaboration  and  will  also  continue  our  work  in 
enlarging  and  managing  the  set  of  objects  in  the  ship  design. 
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Our  objective  in  this  research  is  to  allow  teams  that  are  physically  separated  to 
do  detailed  design  work  on  large-scale,  3D  projects.  We  seek  to  develop  new 
methods  to  facilitate  cooperative  remote  design  utilizing  both  high-  bandwidth 
networking  capability  and  virtual  reality  with  appropriate  graphical  interfaces  to 
support  the  collaborative  effort.  The  specific  task  is  to  enable  multiple  users  to 
stand  at  each  of  two  (or  more)  Virtual  Workbenches  at  remote  locations  and 
interact  effectively  for  design  generation.  Although  we  are  just  in  the  first  year  of 
this  effort,  we  have  made  significant  progress  towards  this  goal. 

To  build  and  test  our  design  environment  we  needed  a  large  scale  and  complex 
design  model.  We  chose  the  new  Navy  Arsenal  Ship.  This  is  a  multiple  deck 
level  model  and  our  biggest  initial  problem  was  to  organize  and  segment  this 
model  so  that  graphical  detail  could  be  handled  efficiently  and  so  that  Individual 
3D  objects  could  be  moved  independently.  We  separated  the  model 
hierarchically  along  decks  and  then  further  along  task  areas  (e.g.,  galley,  mess 
area,  engine  room,  communications  rooms,  sleeping  quarters,  etc.).  Initially,  we 
implemented  the  model  using  the  Simple  Virtual  Environment  (SVE),  which 
provides  extensive  software  support  for  the  creation  of  virtual  applications 
including  handling  tracking,  interactive  tools,  virtual  menus,  and  display  of  3D 
objects  within  the  virtual  environment.  During  the  Summer  of  1996  we  switched 
from  SVE  to  the  SGI  Performer  environment.  The  main  reasons  for  this  is  that 
Performer  is  highly  optimized  for  running  3D  interactive  simulations  on  Reality 
Engines,  Infinite  Reality  Engines,  and  other  high-end  SGI  hardware,  and  it  has 
recently  been  made  more  flexible  with  new  capabilities.  Its  optimizations 
including  advanced  handling  of  graphical  details  and  textures  to  sustain 
interactivity  and  use  of  parallelism  in  multiprocessor  architectures.  New 
capabilities  include  the  handling  of  over  30  model  formats  and  integration  with 
Multigen  modeling  capacities. 

After  this  initial  work  we  have: 

•  Developed  off-axis  stereoscopic  display  and  basic  tracking  mechanisms 
within  the  Performer  environment. 

•  Developed  an  object  data  organization  for  the  design  environment.  This 
involved  labeling  by  base  element,  function,  deck,  size,  modular  grouping, 
and  other  attributes.  With  this  structure  we  can  locate  and  place  objects, 
group  them,  evaluate  their  fit  and  use,  and  provide  input  to  automatically 
update  CAD  databases. 

•  We  have  generated  nearly  1 ,000  3D  objects  and  classified  them  using  the 
data  organization  format.  These  are  items  for  the  living  quarters,  galley,  and 
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mess  areas,  among  others.  We  or  others  can  now  generate  additional 
objects  in  Multigen,  3D  studio.  Wavefront,  or  other  formats  accepted  by 
Performer  or  Multigen. 

•  Used  the  data  organization,  to  develop  models  for  a  number  of  objects  in  the 
cargo  and  engine  rooms,  living  quarters,  mess  hall,  galley,  passageways, 
and  other  areas  on  all  decks  of  the  ship.  These  have  now  been  converted 
into  the  Multigen  format. 

•  Worked  on  several  interaction  paradigms,  many  of  which  are  implemented 
in  the  design  environment.  These  include  two-handed  manipulation  to  grab, 
orient,  and  stretch  or  compress  objects.  Also,  we  have  an  interface  for  pinch 
gloves,  whereby  separate  gestures  involving  one  finger  touching  any  other 
can  be  accurately  recognized,  and  a  serial  interface  for  a  tool  handle  that 
has  multiple  buttons  and  can  be  programmed  for  different  purposes. 

•  Developed  a  networking  structure  that  supports  the  collaborative  virtual 
environment  by  allowing  all  objects  in  the  environment  to  be  "shared".  By 
this  we  mean  that  each  user  in  the  VE  will  be  able  to  see  all  objects  (models) 
in  the  environment  and  any  changes  made  to  those  objects.  If  one  user 
moves  a  chair,  the  chair  will  move  for  all  other  users,  keeping  each  user's 
view  of  that  chair  (position,  orientation,  etc.)  consistent  with  everyone  else's. 
Users  will  also  see  graphical  representations  (avatars)  of  the  other  users  in 
the  environment.  These  avatars  are  themselves  "shared"  objects. 

We  have  now  put  together  and  are  demonstrating  a  prototype  design 
environment.  Based  on  Performer,  OpenGL,  and  Multigen,  it  is  flexible  enough 
for  integration  with  simulations  and  interfaces  supplied  by  Lockheed-Martin  or 
DARPA.  It  can  work  in  networked  environments  involving  multiple 
workbenches,  workbenches  and  displays,  HMD  systems  and  workbenches,  or 
HMD  systems  and  displays. 

It  allows: 

•  Participants  to  see  representations  (avatars)  of  all  users  interacting  in  the 
environment.  The  avatars  will  vary  depending  on  the  type  of  environment. 
For  example,  in  the  networked  workbench  environment,  one  might  just  see 
representations  of  collaborators'  hands.  In  an  environment  involving  a 
workbench  and  HMD,  the  person  in  the  HMD  would  be  immersed  and  might 
see  only  her  collaborator's  hand  while  the  person  at  the  workbench  would 
see  an  avatar.  The  avatar  would  have  the  position  and  orientation  of  the 
immersed  person  in  the  HMD. 

•  Changes  made  to  the  model  to  be  communicated  to  each  participant 
allowing  them  to  see  an  up-to-date  representation  of  objects  in  the 
environment. 
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•  Participants  to  communicate  with  each  other  through  a  digital  voice  channel. 
While  this  is  not  absolutely  necessary,  we  believe  that  it  greatly  enhances 
the  collaboration. 

•  Users  to  perform  simple  manipulations  (translation,  rotation,  simple  grouped 
actions)  with  the  individual  objects  comprising  the  ship  model. 


FURTHER  WORK 

Now  that  we  have  our  initial  system  in  place,  we  will  coordinate  with  Lockheed- 
Martin's  simulation-based  design  efforts.  This  will  include  working  with 
Division,  which  has  been  chosen  to  interface  its  dVISE  virtual  prototyping 
software  with  HLA.  We  will  direct  efforts  towards  integrating  dVISE  into  our 
collaborative  environment. 

Georgia  Tech  has  recently  won  a  DURIP  grant  and  will  use  this  funding  plus 
significant  matching  funds  to  obtain  its  own  immersive  workbench  setup.  This 
facility  will  be  compatible  with  the  NRL  system.  At  that  time  we  will  begin  a  joint 
study  involving  two  workbenches  and  a  long-distance,  networked  environment. 
We  will  further  investigate  setting  up  a  link  between  DARPA  and  NRL  for 
demonstration  and  further  studies  of  collaboration.  This  will  permit  us  to  study 
an  environment  with  more  than  two  sites  participating  and  with  one  or  more 
collaborators  at  each  site. 

We  will  further  study  issues  of  collaboration.  Working  these  out  will  be 
especially  important  when  we  have  multiple,  networked  users.  There  must,  for 
example,  be  support  for  "group  memory"  since  the  design  projects  will  be 
spatially  complex  with  multiple  steps  and  multiple  sessions.  The  collaborators 
must  have  support  to  recall  what  they  did  both  in  individual  sessions  and  across 
sessions.  In  addition  we  must  work  out  protocols  for  collaboration  to  reduce 
latency  of  interaction  and  induce  efficient  sharing  of  tools.  For  example,  we  will 
institute  various  roles  for  collaborators,  such  as  "lecturer  and  audience", 
"apprentice  and  master",  and  "peers",  among  others.  These  roles  range  from 
total  control  of  tools  and  simulation  objects  on  one  end  to  a  free-for-all 
environment  (peers)  where  one  collaborator  can  take  over  control  of  a  tool 
without  permission.  In  addition  to  addressing  control  and  style  of  work  in 
different  ways,  these  protocols  have  different  impacts  on  latency  since  the 
imposing  and  removing  of  locks  on  tools  (and  the  broadcasting  of  this 
information)  takes  time  and  can  interrupt  the  flow  of  interaction. 

In  line  with  our  study  of  the  multi-user,  collaborative  environment  and  issues  of 
data  transfer  and  latency,  we  will  implement  a  robust  networked  structure.  This 
will  be  based  on  work  we  are  doing  now,  in  conjunction  with  the  Georgia  Tech 
Telecommunications/  Networking  Research  Group  and  NRL.  (Our  present 
networked  interface  is  based  on  this  work.)  The  networked  structure  is  based 
on  the  concepts  of  dynamically  configurable  protocols  and  adaptable 
applications.  It  will  dynamically  adjust  to  current  network  conditions,  sacrificing 
speed  for  reliability  (or  vice-versa)  based  on  the  type  of  data.  For  example,  for 
data  from  an  electromagnetic  tracker,  it's  important  to  transmit  the  data  as  fast 
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as  possible,  but  it's  no  big  loss  if  an  occasional  transmission  doesn't  make  it. 
Therefore  the  reliability  of  the  transmission  can  be  sacrificed  in  order  to  reduce 
the  latency.  However,  suppose  the  actual  structure  of  a  model  is  changed?  It's 
important  that  all  users  be  notified  of  this  change,  making  the  reliability  of  the 
transmission  more  important  than  the  latency.  We  will  be  able  to  configure  the 
networking  in  our  application  to  distinguish  between  these  types  of 
transmissions. 

We  will  continue  looking  at  HLA  and  getting  advice  on  compliance  from  experts 
at  Georgia  Tech,  NRL,  Lockheed-Martin,  and  Division.  At  the  moment  HLA 
appears  to  be  just  in  a  prototyping  phase;  our  initial  task  has  been  to  develop 
an  environment  flexible  enough  to  accept  HLA  when  a  final  form  is  decided 
upon.  When  HLA  development  moves  far  enough  along,  we  will  implement  the 
appropriate  interface  for  our  cooperative  remote  design  environment. 

Finally  we  will  continue  our  work  in  enlarging  and  managing  the  set  of  objects 
in  the  ship  design.  This  will  include  further  methods  for  grouping,  re-grouping 
(including  making  new  groups),  and  ungrouping  of  functional  elements  in  the 
design  space.  For  the  more  complicated  objects  and  groups  we  will  introduce 
multiple  levels  of  detail,  which  can  be  handled  in  the  Performer  environment. 
We  will  also  work  on  integrated  detail  management  strategies  involving 
cost/benefit  analyses  to  maintain  interactivity.  (These  are  necessary  when 
handling  views  containing  many  groups  of  objects.)  We  must  also  introduce 
dynamic  objects,  which  allow  changes  to  object  structure  or  function  in  a 
simulation-based  environment.  This  will  necessitate  efficient  organization, 
passing,  and  incorporation  of  updates  to  object  attributes  (beyond  position  and 
orientation)  between  collaborators. 
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